{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Use Ragas to evaluate RAG pipeline\n",
    "\n",
    "Ragas is an open source project for evaluating RAG components.\n",
    "\n",
    "<div>\n",
    "<img src=\"./ragas_eval_image.png\" width=\"80%\"/>\n",
    "</div>\n",
    "\n",
    "**Please note that RAGAS can use a large amount of OpenAI api token consumption.** <br> \n",
    "\n",
    "Read through this notebook carefully and pay attention to the number of questions and metrics you want to evaluate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 1. Prepare Ragas environment and ground truth data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# ! python -m pip install openai dataset ragas langchain pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Question</th>\n",
       "      <th>ground_truth_answer</th>\n",
       "      <th>OpenAI_RAG_answer</th>\n",
       "      <th>Custom_RAG_answer</th>\n",
       "      <th>Custom_RAG_context</th>\n",
       "      <th>Uri</th>\n",
       "      <th>H1</th>\n",
       "      <th>H2</th>\n",
       "      <th>Score</th>\n",
       "      <th>Reason</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>What do the parameters for HNSW mean?\\n</td>\n",
       "      <td>M: maximum out-degree of the graph in a layer;...</td>\n",
       "      <td>The HNSW parameters include the “nlist” which ...</td>\n",
       "      <td>The parameters for HNSW are as follows:\\n- M: ...</td>\n",
       "      <td>'''the next layer to begin another search. Aft...</td>\n",
       "      <td>https://pymilvus.readthedocs.io/en/latest/para...</td>\n",
       "      <td>Index</td>\n",
       "      <td>Milvus support to create index to accelerate v...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>What are HNSW good default parameters when dat...</td>\n",
       "      <td>M=16, efConstruction=32, ef=32</td>\n",
       "      <td>The default HNSW parameters for data size of 2...</td>\n",
       "      <td>A good default value for the HNSW parameters w...</td>\n",
       "      <td>'''to reduce the probability that the target v...</td>\n",
       "      <td>https://pymilvus.readthedocs.io/en/latest/para...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>what is the default distance metric used in AU...</td>\n",
       "      <td>Trick answer:  IP inner product, not yet updat...</td>\n",
       "      <td>The default AUTOINDEX distance metric in Milvu...</td>\n",
       "      <td>The default AUTOINDEX distance metric in Milvu...</td>\n",
       "      <td>'''please refer to Milvus documentation index ...</td>\n",
       "      <td>https://pymilvus.readthedocs.io/en/latest/tuto...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>How did New York City get its name?</td>\n",
       "      <td>In the 1600’s, the Dutch planted a trading pos...</td>\n",
       "      <td>I'm sorry, but I couldn't find any information...</td>\n",
       "      <td>New York City was originally named New Amsterd...</td>\n",
       "      <td>'''Etymology See also: Nicknames of New York C...</td>\n",
       "      <td>https://en.wikipedia.org/wiki/New_York_City</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Question  \\\n",
       "0            What do the parameters for HNSW mean?\\n   \n",
       "1  What are HNSW good default parameters when dat...   \n",
       "2  what is the default distance metric used in AU...   \n",
       "3                How did New York City get its name?   \n",
       "\n",
       "                                 ground_truth_answer  \\\n",
       "0  M: maximum out-degree of the graph in a layer;...   \n",
       "1                     M=16, efConstruction=32, ef=32   \n",
       "2  Trick answer:  IP inner product, not yet updat...   \n",
       "3  In the 1600’s, the Dutch planted a trading pos...   \n",
       "\n",
       "                                   OpenAI_RAG_answer  \\\n",
       "0  The HNSW parameters include the “nlist” which ...   \n",
       "1  The default HNSW parameters for data size of 2...   \n",
       "2  The default AUTOINDEX distance metric in Milvu...   \n",
       "3  I'm sorry, but I couldn't find any information...   \n",
       "\n",
       "                                   Custom_RAG_answer  \\\n",
       "0  The parameters for HNSW are as follows:\\n- M: ...   \n",
       "1  A good default value for the HNSW parameters w...   \n",
       "2  The default AUTOINDEX distance metric in Milvu...   \n",
       "3  New York City was originally named New Amsterd...   \n",
       "\n",
       "                                  Custom_RAG_context  \\\n",
       "0  '''the next layer to begin another search. Aft...   \n",
       "1  '''to reduce the probability that the target v...   \n",
       "2  '''please refer to Milvus documentation index ...   \n",
       "3  '''Etymology See also: Nicknames of New York C...   \n",
       "\n",
       "                                                 Uri     H1  \\\n",
       "0  https://pymilvus.readthedocs.io/en/latest/para...  Index   \n",
       "1  https://pymilvus.readthedocs.io/en/latest/para...    NaN   \n",
       "2  https://pymilvus.readthedocs.io/en/latest/tuto...    NaN   \n",
       "3        https://en.wikipedia.org/wiki/New_York_City    NaN   \n",
       "\n",
       "                                                  H2  Score  Reason  \n",
       "0  Milvus support to create index to accelerate v...    NaN     NaN  \n",
       "1                                                NaN    NaN     NaN  \n",
       "2                                                NaN    NaN     NaN  \n",
       "3                                                NaN    NaN     NaN  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Read questions and ground truth answers into a pandas dataframe.\n",
    "# Note: Surround each context string with ''' to avoid issues with quotes inside.\n",
    "# Note: Separate each context string with a comma.\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# Read ground truth answers from file.\n",
    "eval_df = pd.read_csv(\"../../../christy_coding_scratch/data/milvus_ground_truth.csv\", \n",
    "                      header=0, skip_blank_lines=True)\n",
    "display(eval_df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ragas default uses HuggingFace Datasets.\n",
    "# https://docs.ragas.io/en/latest/getstarted/evaluation.html\n",
    "from datasets import Dataset\n",
    "\n",
    "def assemble_ragas_dataset(input_df):\n",
    "    \"\"\"Assemble a RAGAS HuggingFace Dataset from an input pandas df.\"\"\"\n",
    "\n",
    "    # Assemble Ragas lists: questions, ground_truth_answers, retrieval_contexts, and RAG answers.\n",
    "    question_list, truth_list, context_list = [], [], []\n",
    "\n",
    "    # Get all the questions.\n",
    "    question_list = input_df.Question.to_list()\n",
    "\n",
    "    # Get all the ground truth answers.\n",
    "    truth_list = input_df.ground_truth_answer.to_list()\n",
    "\n",
    "    # Get all the Milvus Retrieval Contexts as list[list[str]]\n",
    "    context_list = input_df.Custom_RAG_context.to_list()\n",
    "    context_list = [[context] for context in context_list]\n",
    "\n",
    "    # Get all the RAG answers based on contexts.\n",
    "    rag_answer_list = input_df.Custom_RAG_answer.to_list()\n",
    "\n",
    "    # Create a HuggingFace Dataset from the ground truth lists.\n",
    "    ragas_ds = Dataset.from_dict({\"question\": question_list,\n",
    "                            \"contexts\": context_list,\n",
    "                            \"answer\": rag_answer_list,\n",
    "                            \"ground_truth\": truth_list\n",
    "                            })\n",
    "    return ragas_ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['question', 'contexts', 'answer', 'ground_truth'],\n",
       "    num_rows: 4\n",
       "})"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create a Ragas HuggingFace Dataset from the ground truth lists.\n",
    "ragas_input_ds = assemble_ragas_dataset(eval_df)\n",
    "display(ragas_input_ds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>question</th>\n",
       "      <th>contexts</th>\n",
       "      <th>answer</th>\n",
       "      <th>ground_truth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>What do the parameters for HNSW mean?\\n</td>\n",
       "      <td>['''the next layer to begin another search. Af...</td>\n",
       "      <td>The parameters for HNSW are as follows:\\n- M: ...</td>\n",
       "      <td>M: maximum out-degree of the graph in a layer;...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>What are HNSW good default parameters when dat...</td>\n",
       "      <td>['''to reduce the probability that the target ...</td>\n",
       "      <td>A good default value for the HNSW parameters w...</td>\n",
       "      <td>M=16, efConstruction=32, ef=32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>what is the default distance metric used in AU...</td>\n",
       "      <td>['''please refer to Milvus documentation index...</td>\n",
       "      <td>The default AUTOINDEX distance metric in Milvu...</td>\n",
       "      <td>Trick answer:  IP inner product, not yet updat...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>How did New York City get its name?</td>\n",
       "      <td>['''Etymology See also: Nicknames of New York ...</td>\n",
       "      <td>New York City was originally named New Amsterd...</td>\n",
       "      <td>In the 1600’s, the Dutch planted a trading pos...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            question  \\\n",
       "0            What do the parameters for HNSW mean?\\n   \n",
       "1  What are HNSW good default parameters when dat...   \n",
       "2  what is the default distance metric used in AU...   \n",
       "3                How did New York City get its name?   \n",
       "\n",
       "                                            contexts  \\\n",
       "0  ['''the next layer to begin another search. Af...   \n",
       "1  ['''to reduce the probability that the target ...   \n",
       "2  ['''please refer to Milvus documentation index...   \n",
       "3  ['''Etymology See also: Nicknames of New York ...   \n",
       "\n",
       "                                              answer  \\\n",
       "0  The parameters for HNSW are as follows:\\n- M: ...   \n",
       "1  A good default value for the HNSW parameters w...   \n",
       "2  The default AUTOINDEX distance metric in Milvu...   \n",
       "3  New York City was originally named New Amsterd...   \n",
       "\n",
       "                                        ground_truth  \n",
       "0  M: maximum out-degree of the graph in a layer;...  \n",
       "1                     M=16, efConstruction=32, ef=32  \n",
       "2  Trick answer:  IP inner product, not yet updat...  \n",
       "3  In the 1600’s, the Dutch planted a trading pos...  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Debugging inspect all the data.\n",
    "ragas_input_df = ragas_input_ds.to_pandas()\n",
    "display(ragas_input_df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 2. Start Ragas Evaluation with custom Evaluation LLM\n",
    "\n",
    "The default OpenAI model used by Ragas is `gpt-3.5-turbo-16k`.\n",
    "\n",
    "Note that a large amount of OpenAI api token is consumed. Every time you ask a question and every evaluation, you will ask the OpenAI service. Please pay attention to your token consumption. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, openai, pprint\n",
    "from openai import OpenAI\n",
    "\n",
    "# Save api key in env variable.\n",
    "# https://help.openai.com/en/articles/5112595-best-practices-for-api-key-safety\n",
    "openai_api_key=os.environ['OPENAI_API_KEY']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Choose the metrics you want to see.\n",
    "# Remove context relevancy metric - it is deprecated and not maintained.\n",
    "from ragas.metrics import (\n",
    "    context_recall, \n",
    "    context_precision, \n",
    "    faithfulness, \n",
    "    # answer_relevancy, \n",
    "    # answer_similarity,\n",
    "    )\n",
    "metrics = ['context_recall', 'context_precision', 'faithfulness']\n",
    "\n",
    "# Change the llm-as-critic.\n",
    "# It is also possible to switch out a HuggingFace open LLM here if you want.\n",
    "# https://docs.ragas.io/en/stable/howtos/customisations/bring-your-own-llm-or-embs.html\n",
    "from ragas.llms import llm_factory\n",
    "LLM_NAME = \"gpt-3.5-turbo\"\n",
    "# Default temperature = 1e-8\n",
    "ragas_llm = llm_factory(model=LLM_NAME)\n",
    "\n",
    "# Also change the embeddings using HuggingFace models.\n",
    "from langchain_community.embeddings import HuggingFaceEmbeddings\n",
    "from ragas.embeddings import LangchainEmbeddingsWrapper\n",
    "EMB_NAME = \"BAAI/bge-large-en-v1.5\"\n",
    "lc_embeddings = HuggingFaceEmbeddings(model_name=EMB_NAME)\n",
    "\n",
    "# # Alternatively use OpenAI embedding models.\n",
    "# # https://openai.com/blog/new-embedding-models-and-api-updates\n",
    "# from langchain_openai.embeddings import OpenAIEmbeddings\n",
    "# lc_embeddings = OpenAIEmbeddings(\n",
    "#     model=\"text-embedding-3-small\", \n",
    "#     # 512 or 1536 possible for 3-small\n",
    "#     # 256, 1024, or 3072 for 3-large\n",
    "#     dimensions=512)\n",
    "ragas_emb = LangchainEmbeddingsWrapper(embeddings=lc_embeddings)\n",
    "\n",
    "# Change the default models used for each metric.\n",
    "for metric in metrics:\n",
    "    globals()[metric].llm = ragas_llm\n",
    "    globals()[metric].embeddings = ragas_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3e03784320a9421081c35b22d09964a7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Evaluating:   0%|          | 0/12 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Invalid JSON response. Expected dictionary with key 'Attributed'\n",
      "Invalid JSON response. Expected dictionary with key 'Attributed'\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>question</th>\n",
       "      <th>contexts</th>\n",
       "      <th>answer</th>\n",
       "      <th>ground_truth</th>\n",
       "      <th>context_precision</th>\n",
       "      <th>context_recall</th>\n",
       "      <th>faithfulness</th>\n",
       "      <th>context_f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>What do the parameters for HNSW mean?\\n</td>\n",
       "      <td>['''the next layer to begin another search. Af...</td>\n",
       "      <td>The parameters for HNSW are as follows:\\n- M: ...</td>\n",
       "      <td>M: maximum out-degree of the graph in a layer;...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>What are HNSW good default parameters when dat...</td>\n",
       "      <td>['''to reduce the probability that the target ...</td>\n",
       "      <td>A good default value for the HNSW parameters w...</td>\n",
       "      <td>M=16, efConstruction=32, ef=32</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>what is the default distance metric used in AU...</td>\n",
       "      <td>['''please refer to Milvus documentation index...</td>\n",
       "      <td>The default AUTOINDEX distance metric in Milvu...</td>\n",
       "      <td>Trick answer:  IP inner product, not yet updat...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>How did New York City get its name?</td>\n",
       "      <td>['''Etymology See also: Nicknames of New York ...</td>\n",
       "      <td>New York City was originally named New Amsterd...</td>\n",
       "      <td>In the 1600’s, the Dutch planted a trading pos...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            question  \\\n",
       "0            What do the parameters for HNSW mean?\\n   \n",
       "1  What are HNSW good default parameters when dat...   \n",
       "2  what is the default distance metric used in AU...   \n",
       "3                How did New York City get its name?   \n",
       "\n",
       "                                            contexts  \\\n",
       "0  ['''the next layer to begin another search. Af...   \n",
       "1  ['''to reduce the probability that the target ...   \n",
       "2  ['''please refer to Milvus documentation index...   \n",
       "3  ['''Etymology See also: Nicknames of New York ...   \n",
       "\n",
       "                                              answer  \\\n",
       "0  The parameters for HNSW are as follows:\\n- M: ...   \n",
       "1  A good default value for the HNSW parameters w...   \n",
       "2  The default AUTOINDEX distance metric in Milvu...   \n",
       "3  New York City was originally named New Amsterd...   \n",
       "\n",
       "                                        ground_truth  context_precision  \\\n",
       "0  M: maximum out-degree of the graph in a layer;...                1.0   \n",
       "1                     M=16, efConstruction=32, ef=32                0.0   \n",
       "2  Trick answer:  IP inner product, not yet updat...                1.0   \n",
       "3  In the 1600’s, the Dutch planted a trading pos...                1.0   \n",
       "\n",
       "   context_recall  faithfulness  context_f1  \n",
       "0             0.0      1.000000         0.0  \n",
       "1             1.0      0.333333         0.0  \n",
       "2             1.0      0.500000         1.0  \n",
       "3             0.0      0.333333         0.0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Evaluate the dataset.\n",
    "from ragas import evaluate\n",
    "\n",
    "ragas_result = evaluate(\n",
    "    ragas_input_ds,\n",
    "    metrics=[\n",
    "        context_precision,\n",
    "        context_recall,\n",
    "        faithfulness,\n",
    "    ],\n",
    "    llm=ragas_llm,\n",
    ")\n",
    "\n",
    "# View evaluations.\n",
    "ragas_output_df = ragas_result.to_pandas()\n",
    "# Calculate average context scores.\n",
    "temp = ragas_output_df.fillna(0.0)\n",
    "temp['context_f1'] = 2.0 * temp.context_precision * temp.context_recall \\\n",
    "                    / (temp.context_precision + temp.context_recall)\n",
    "temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using 4 eval questions, Mean Retrieval F1 Score = 0.25\n"
     ]
    }
   ],
   "source": [
    "# Display Retrieval average score.\n",
    "print(f\"Using {eval_df.shape[0]} eval questions, Mean Retrieval F1 Score = {np.round(temp.context_f1.mean(),2)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Author: Christy Bergman\n",
      "\n",
      "Python implementation: CPython\n",
      "Python version       : 3.11.7\n",
      "IPython version      : 8.21.0\n",
      "\n",
      "datasets : 2.18.0\n",
      "langchain: 0.1.11\n",
      "openai   : 1.13.3\n",
      "ragas    : 0.1.5\n",
      "\n",
      "conda environment: py311\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Props to Sebastian Raschka for this handy watermark.\n",
    "# !pip install watermark\n",
    "\n",
    "%load_ext watermark\n",
    "%watermark -a 'Christy Bergman' -v -p datasets,langchain,openai,ragas --conda"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
